Skip to content

[SPARK-10071][Streaming]Output a warning when writing QueueInputDStream and throw a better exception when reading QueueInputDStream#8624

Closed
zsxwing wants to merge 1 commit into
apache:masterfrom
zsxwing:SPARK-10071
Closed

[SPARK-10071][Streaming]Output a warning when writing QueueInputDStream and throw a better exception when reading QueueInputDStream#8624
zsxwing wants to merge 1 commit into
apache:masterfrom
zsxwing:SPARK-10071

Conversation

@zsxwing

@zsxwing zsxwing commented Sep 6, 2015

Copy link
Copy Markdown
Member

Output a warning when serializing QueueInputDStream rather than throwing an exception to allow unit tests use it. Moreover, this PR also throws an better exception when deserializing QueueInputDStream to make the user find out the problem easily. The previous exception is hard to understand: https://issues.apache.org/jira/browse/SPARK-8553

@SparkQA

SparkQA commented Sep 6, 2015

Copy link
Copy Markdown

Test build #42075 has finished for PR 8624 at commit 847cfa8.

  • This patch fails PySpark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@zsxwing

zsxwing commented Sep 7, 2015

Copy link
Copy Markdown
Member Author

retest this please

@SparkQA

SparkQA commented Sep 7, 2015

Copy link
Copy Markdown

Test build #42080 has finished for PR 8624 at commit 847cfa8.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@tdas

tdas commented Sep 9, 2015

Copy link
Copy Markdown
Contributor

LGTM. Merging this to master, 1.5 and 1.4.

asfgit pushed a commit that referenced this pull request Sep 9, 2015
…ream and throw a better exception when reading QueueInputDStream

Output a warning when serializing QueueInputDStream rather than throwing an exception to allow unit tests use it. Moreover, this PR also throws an better exception when deserializing QueueInputDStream to make the user find out the problem easily. The previous exception is hard to understand: https://issues.apache.org/jira/browse/SPARK-8553

Author: zsxwing <zsxwing@gmail.com>

Closes #8624 from zsxwing/SPARK-10071 and squashes the following commits:

847cfa8 [zsxwing] Output a warning when writing QueueInputDStream and throw a better exception when reading QueueInputDStream

(cherry picked from commit 820913f)
Signed-off-by: Tathagata Das <tathagata.das1565@gmail.com>
asfgit pushed a commit that referenced this pull request Sep 9, 2015
…ream and throw a better exception when reading QueueInputDStream

Output a warning when serializing QueueInputDStream rather than throwing an exception to allow unit tests use it. Moreover, this PR also throws an better exception when deserializing QueueInputDStream to make the user find out the problem easily. The previous exception is hard to understand: https://issues.apache.org/jira/browse/SPARK-8553

Author: zsxwing <zsxwing@gmail.com>

Closes #8624 from zsxwing/SPARK-10071 and squashes the following commits:

847cfa8 [zsxwing] Output a warning when writing QueueInputDStream and throw a better exception when reading QueueInputDStream

(cherry picked from commit 820913f)
Signed-off-by: Tathagata Das <tathagata.das1565@gmail.com>
@asfgit asfgit closed this in 820913f Sep 9, 2015
@zsxwing zsxwing deleted the SPARK-10071 branch September 9, 2015 08:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants